Fraud Detection

About Fraud Detection

Some of the biggest threats to data quality are bots and cheaters. Often bots will complete surveys en masse, or a person will take surveys on behalf of someone else multiple times. Thankfully, Expert Review tracks these patterns, so you can ensure your surveys are collecting only the highest quality of data.

With Expert Review Fraud Detection, you can do the following with responses identified as fraudulent:

• Discard them, preventing them from being counted against auditable responses or quotas.
• Redirect these responses for analysis separately.
• Flag these responses so they can be filtered, reported on, etc.
• Analyze the number of fraudulent responses you’ve received, breaking them down by duplicates and bots.

You can also prevent email scanning software from inadvertently submitting a survey response to begin with.

You can enable Prevent multiple submissions, Bot detection, Security scan monitor, and RelevantID all in the same survey.

Enabling Fraud Detection for a User

Before you can enable any of the individual fraud detection settings in your project, you need to have access to the features.

Once Fraud Detection is added to your license, a Brand Administrator can adjust the following permissions for a user, user type, or group:

• RelevantID
• Bot Detection (security scan monitor)
• Enhanced Ballot Box Stuffing (prevent multiple submissions)

Prevent Multiple Submissions

When enabled in your survey options, Prevent multiple submissions allows you to detect duplicates and keep them from making their way through the survey. During the first survey session, Prevent multiple submissions places a cookie on the browser. If the same respondent comes back on the same browser and device, without having cleared their cookies, they are flagged as a duplicate.

Enabling Prevent Multiple Submissions

1. Go to Survey options.
   
2. Go to Security.
3. Enable Prevent multiple submissions.
4. Choose one of the following actions:
   • End survey with a message: Once a respondent is determined to have already taken the survey, the respondent is sent out of the survey and we display an end of survey message. Since the survey ends before the respondent can fill it out, you will not have any data this duplicate respondent might produce.
   • Redirect to URL: Once a respondent is determined to have already taken the survey, immediately redirect them out of the survey to a URL. Since the survey ends before the respondent can fill it out, you will not have any data this duplicate respondent might produce.
     Qtip: This feature is not available for free accounts.
   • Flag responses: Duplicate respondents will not be sent out of the survey, and will be allowed to finish their response. However, their responses will be assigned a value under the field Q_BallotBoxStuffing.
5. If you selected End survey with a message, you’ll be asked whether you want to use the default end of survey message, or select a custom one. You can write a new message or use one saved in your library in the past.
   
6. If you selected Redirect to URL, provide the full URL of the website you’d like to redirect respondents to.
   
7. After your survey is completed and before you are ready to distribute to respondents, click Publish.
   Qtip: Save this step for when you are done setting up your survey!

Q_BallotBoxStuffing

When you select Continue survey and set embedded data field, respondents will not be sent out of the survey, but their response will be recorded and be assigned a value under the field Q_BallotBoxStuffing. Here’s how you interpret this field:

You can report on or analyze Q_BallotBoxStuffing without adding it to the survey flow. However, in order to set up logic based on this field (such as screening out possible bots), you must add this embedded data field to your survey flow.

Bot Detection

Bot detection allows you to track which responses are likely bots by adding a field to each response called Q_RecaptchaScore. Every response is rated on the probability that the respondent was a bot, which you can then use to filter out data or build reports.

Enabling Bot Detection

1. Go to Survey options.
2. Go to Security.
3. Enable Bot Detection.
4. After your survey is completed and before you are ready to distribute to respondents, click Publish.
   Qtip: Save this step for when you are done setting up your survey!

Q_RecaptchaScore

When you enable bot detection, this activates the field Q_RecaptchaScore, which can be used in reporting to indicate whether a response is more likely a bot or a human. This field uses Google’s invisible reCaptcha technology.

You can report on or analyze Q_RecaptchaScore without adding it to the survey flow. However, in order to set up logic based on this field (such as screening out possible bots), you must add this embedded data field to your survey flow.

Security Scan Monitor

When enabled in your survey options, security scan monitor allows you to prevent email scanning software from inadvertently starting a survey session when a survey link is included in the email. While this feature is designed to stop email scanning software from opening emailed links, it will apply to links of any type, regardless if they were distributed via Qualtrics or a third-party system.

How Security Scan Monitor Works

Bot detection is used to determine whether or not a roadblock should be placed at the start of a survey, preventing the email scanning software from starting a survey session. If a bot is detected, a start page will be shown instead of the first question in the survey. If a bot was incorrectly detected, the customer can click the “next” button and be able to continue taking the survey. The look and feel of the survey start page will be consistent with what has been selected from the rest of the survey.

Enabling Security Scan Monitor

1. Go to Survey options.
2. Go to Security.
3. Enable Security Scan Monitor.
4. After your survey is completed and before you are ready to distribute to respondents, click Publish.

RelevantID

RelevantID improves fraud detection by assessing respondent metadata to determine the likelihood that the same respondent is answering over and over; RelevantID does not necessarily check the content of the responses for duplicates, since respondent can answer multiple times while giving different answers. This feature has four fields associated with it that you can report on: Q_RelevantIDDuplicate, Q_RelevantIDDuplicateScore, Q_RelevantIDFraudScore, and Q_RelevantIDLastStartDate. These fields are calculated using RelevantID technology. This technology checks if the respondent is cheating by taking the survey multiple times or whether a survey taker is fraudulent by analyzing a user’s browser, operating system, and location to provide a fraud score. See RelevantID‘s site for more details.

Enabling RelevantID

1. Go to Survey options.
2. Go to Security.
3. Enable RelevantID.
4. Set the timeframe during which RelevantID resets and will not flag a response that seems to be a duplicate from a previous session. Your options are 1 week, 30 days, 3 months, 6 months, or 1 year.
   Qtip: This setting is helpful if you need to let the same respondents retake a survey after a period of time. This way, if you are flagging and possibly screening out duplicate respondents with RelevantID, you do not have to change your Survey Flow setup when it comes time for the respondents to retake the survey.
5. After your survey is completed and before you are ready to distribute to respondents, click Publish.
   Qtip: Save this step for when you are done setting up your survey!

Q_RelevantIDDuplicate, Q_RelevantIDDuplicateScore, Q_RelevantIDFraudScore, and Q_RelevantIDLastStartDate

You can report on or analyze these fields without adding them to the survey flow. However, in order to set up logic based on these fields (such as screening out fraudulent responses), you must add this embedded data field to your survey flow.

Adding Fraudulent Detection Fields to the Survey Flow

You can report on or analyze Fraud Detection fields like Q_RelevantIDDuplicate, Q_RelevantIDDuplicateScore, Q_RelevantIDFraudScore, Q_RecaptchaScore, and Q_BallotBoxStuffing simply by having the corresponding survey option enabled, without adding them to the survey flow.

However, in order to set up logic based on these fields (such as screening out possible bots or fraudulent responses), you must add these embedded data fields to your survey flow.

1. In the Survey options, turn on Prevent multiple submissions (Flag responses), Bot detection, and / or RelevantID.
   
2. Go to the Survey flow.
   
3. Click Add a New Element Here.
4. Select Embedded Data.
5. Select the name of the embedded data, click the blue dropdown, highlight Survey Metadata, and select the desired fraudulent response metrics.
   
   Qtip: Q_BallotBoxStuffing is under “Ballot Box Stuffing.”
6. Move the embedded data to the top of the survey flow.
   
7. Click Apply.

You can now build your desired branch logic.

Example: Use these embedded data fields and the values below to redirect fraudulent respondents using branch logic. Note how the block appears before the branch logic.